1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and a program.
2. Description of the Related Art
Remote operation of consumer electronics (hereinafter, “CE appliances”) such as a mobile music player and a digital photo viewer is typically carried out by a user operating a remote controller. However, with this method, an apparatus called a “remote controller” is required. For this reason, there is also a technology for enabling a user to operate an appliance by way of gestures made by the user's body and/or hands without using a remote controller.
For example, there is a technology that uses 3D sensors, which are capable of acquiring depth information by emitting infrared light and measuring reflected light, to detect a region corresponding to a person and body movements in the reflected image. There is also a technology that uses a normal camera and mainly recognizes moving regions in video images as body movements of a person. As one example, video games that apply such technologies have been commercialized.
As examples, Japanese Laid-Open Patent Publication Nos. 2008-40576 and 2004-78977 and Japanese Laid-Open Patent Publication (Translated PCT Application) No. 2009-519552 disclose technologies for using a normal camera to recognize the shape of a person's hand in particular, and recognizing gestures based on a shape pattern and changes in position of the hand. As an extension to such proposed technologies, technology for recognizing hands or people themselves from video images has been developed, as disclosed in Japanese Laid-Open Patent Publication No. 2009-140283.
There are two methods of recognizing changes in the shapes and/or positions of a person's hands or body in video images as gestures. One method focuses on relative changes in the person's hands or body that have been recognized. As shown in FIG. 11A, the shapes of the hands are recognized and relative changes in the position of the hands are calculated. FIGS. 11A and 11B are diagrams useful in showing the screen of an image displaying apparatus 30, and show a state where a music player is being operated by software. A jacket photograph 55 of a music CD and a window (image frame) 50, in which an image of a user 80 picked up by a camera 10 is displayed, are displayed on the screen of the image displaying apparatus 30.
As shown in FIG. 10, the camera 10 is mounted on the image displaying apparatus 30. FIG. 10 is a diagram useful in showing the relationship between a user 1, the camera 10, and the image displaying apparatus 30. Once the user's hand 82 has been recognized, recognition is continually carried out while the user 1 is performing a gesture, and when the hand is next recognized as the hand 83, the relative change between the hand 82 and the hand 83 is decided as a gesture.
As shown in FIG. 11B, the other method recognizes the shape of a hand 84 and uses the absolute position of the hand 84. The absolute coordinates of an image displayed in the window 50 are used to calculate the coordinate position of the recognized hand 84. The hand 84 in the window 50 then interacts with GUI elements such as a button 52 that are displayed by being overlaid (superimposed) at set positions in the image.
However, when the distance between the camera 10 and the user 1 shown in FIG. 10 changes, the size of the user 80 in the image also changes. That is, if the distance between the camera 10 and the user 1 is short, the user 80 will appear large in the image in the window 50 as shown in FIG. 12A, while if the distance between the camera 10 and the user 1 is far, the user 80 will appear small in the image in the window 50 as shown in FIG. 12B.
For this reason, when the user 1 is at a certain position in front of the camera 10, the range in which the hand 84 may move within the image will differ between when the user 80 appears large on the screen and when the user 80 appears small on the screen. When the user 80 appears large on the screen, as shown in FIG. 12A, the hand 84 is capable of moving in the range 62, but when the user 80 appears small on the screen, as shown in FIG. 12B, the hand 84 is capable of moving only in the range 64. Accordingly, if GUI elements such as buttons displayed by being overlaid are always disposed at the same positions set relative to the window 50, when the user 80 appears small on the screen, the user 1 will need to walk around in front of the camera 10 to touch GUI elements such as the button 52. FIGS. 12A and 12B are diagrams useful in showing images displayed in the window 50.
In Publication No. 2009-519552, although a problem occurs in that the size of the user in the image varies according to changes in the distance between the camera and the user as shown in FIGS. 12A and 12B, no consideration whatsoever is given to this problem. Publication No. 2004-78977 meanwhile does not use an image picked up with a camera as a GUI display. However, there is the problem that the distance that the user needs to move to touch a virtual switch that is a GUI element changes in accordance with the distance between the camera and the user.